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Rationale and 
Process Followed 



• This investigation analyzes historical data 
to identify schedule drivers. 


• Goal is to derive schedule estimating 
relationships (SERs) at the phase level. 

- Phase is defined as the duration between 
major project milestones. 

• This investigation uses a 2-pass approach. 















Data Sources 



• Technical and schedule data used in this 
study came primarily from three sources: 

1 . Rutkowski schedule database 

2. QuickCost database 

3. NAFCOM 2008 database 

• Additional data obtained from the REDSTAR 
library to fill-in missing values. 

https://redstar.saic.com 
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• SERs generated with full mission set for 4 schedule durations 


Phase 

Approach 

Number 
of Points 

F-Test p- 
value 

Pearson's R- 
Sq 

SEE 

Start-PDR 

Multiplicative (Mission Class Avg) 

87 

0.036 

0.274 

0.88 

Multiplicative (Mission Class Trim Mean) 

87 

0.0437 

0.267 

0.881 

Additive 

87 

0.0289 

0.281 

1.22 

PDR-CDR 

Multiplicative (Mission Class Avg) 

82 

0.0121 

0.325 

0.635 

Multiplicative (Mission Class Trim Mean) 

82 

0.0141 

0.32 

0.636 

Additive 

82 

0.0543 

0.275 

1.091 

Start-CDR 

Multiplicative (Mission Class Avg) 

87 

0.0279 

0.282 

0.58 

Multiplicative (Mission Class Trim Mean) 

87 

0.0102 

0.312 

0.623 

Additive 

87 

0.006 

0.327 

1.31 

CDR-Delivery 

Multiplicative (Mission Class Avg) 

61 

<0.0001 

0.628 

0.42 

Multiplicative (Mission Class Trim Mean) 

61 

<0.0001 

0.605 

0.435 

Additive 

61 

0.0132 

0.422 

1.27 


• In these runs, not much difference between multiplicative 
approaches 

• Additive approach as good or worse than multiplicative 

• No appreciable difference with PDR as a milestone 

• No acceptable SERs up to CDR milestone using all missions 




• Therefore, next step was to investigate Mission Class-specific SERs 

- Earth Orbiting (EO) 

- Launch Vehicle/Manned (LV/M) 

- Planetary (PL) 


• This yielded more significant results 


Phase 

Mission Class 

Number 
of Points 

F-Test 

p-value 

Pearson's 

R-Sq 

SEE 

Start-CDR 

(Design) 

Earth Orbiting 

35 

<0.001 

0.826 

0.329 

Launch Vehicle / Manned 

19 

0.005 

0.727 

0.327 

Planetary 

25 

<0.001 

0.804 

0.227 

CDR-Delivery 

(Manufacturing) 

Earth Orbiting 

22 

<0.001 

0.856 

0.306 

Launch Vehicle / Manned 

16 

0.008 

0.821 

0.219 

Planetary 

22 

<0.001 

0.751 

0.301 
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Launch Vehicle/Manned 
SER Regression 


Start _ CDR _ Dur = 377 .877 FundAvaU~°™ ManufMethods' 00 * 9 
StartYr 0 1 29 StreamEM “° 432 R e usable 0393 Crewed 0 925 
PostApollo Re qts° 366 Parallel' 0 5m 

F Test p-value = 0.005 
Pearson R 2 = 0.727 
Est Std Error = 0.327 

CDR Delivery Dur = 13 .1 SZEngrMgmt 0 ' 11 PowerGen 0 ™ 4 
StartYr Crewed"'"' Post Apollo Re qts"‘" Parallel 

F Test p-value = 0.008 
Pearson R 2 = 0.821 
Est Std Error = 0.219 






Independent Variable 
Details 


Indicator? Units 


Mix of indicator and numeric 
variables 

Heritage to NAFCOM 
Management Factor 
definitions 

Complexity Variable is sum 
of normalized Dry Weight, 
Maximum Data Rate, and 
Number of Instruments 

- Aggregated these variables 
to alleviate autocorrelation 
effects 

- Normalized to avoid effects 
of scale 


CommSat 

Yes 


Complexity 

- Dry Wt 

- MaxData 

- Num Instruments 


pounds 

kbps 

Crewed 

Yes 


Design Life 


Months 

DoD-Owned 

Yes 


Engr Mgt 



Funding Avail 



Great Obs Class 

Yes 


Manuf Methods 



Off-the-Shelf Bus 

Yes 


Parallelization 

Yes 


Post-Apollo Man-Rated 
Requirements 

Yes 


Power Generated 


LEO Equiv 
Watts 

Reusable 

Yes 


RTG-Powered 

Yes 


Start Year 


Yr-1960 

Streamlined Engr Mgmt 

Yes 


Test Approach 






Regression Factor 
T rends 


Are there any meaningful trends for SER 
regression factors? 


Project start year is the 
most common factor 

Engineering Mgmt 
significant in some 
capacity for all SERs 

Many class-specific 
factors significant 

Legend 

Significant to SER 
Not significant 
Excluded from Analysis 
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• As a means of validation, the same data was used to 
generate SERs with a different regression method 

- Minimum Unbiased Percent Error (MUPE) selected 


• Results obtained were nearly identical to log- 
transformed ordinary least squares (LOLS) regressions 

- Magnitude of coefficients changed very little — coefficients 
differed by less than 12% 

- Statistical significance very similar 

- Adds credibility to LOLS results 

• Addition verification performed to test fundamental 
assumptions of LOLS regression 



LVM SER Residual 
Analysis — Acceptable 



LVM Design SER 
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Equal Variance Assumption: No significant trend 
evident, assumption valid. 


LVM Manufacturing SER 


Residual Analysis: Fitted Vs. Resid 



Equal Variance Assumption: No significant trend 
evident, assumption valid. 


LVM Design Resid 




Normality 
Assumption: 
Log residuals 
normally 
distributed. 


LVM Prod Resid 



Normality 
Assumption: 
Log residuals 
normally 
distributed. 


Normal Quantile Rot 


Normal Quantie Rot 


\ Design & Manufacturing 
r Correlation 


• Desirable to combine estimated design & 
manufacturing durations. 

- Means sum together 

- Garvey shows that variances sum with covariance factor 

• Analysis shows there is no correlation between the 
design and manufacturing residuals. 

- Pearson’s R 2 
correlation of 0.0007 

- Covariance reduces 
to 0 

• Straight sum of 
variance is 
appropriate. 

Reference: Probability Methods for Cost 
Uncertainty Analysis , Paul Garvey, 1999. 


Residual Correlation Plot 
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Significant Inputs 


SERRA Model- 
Inputs 


Mission Type 


Options 

Launch Vehicle/Manned 



1^ Schedule Risk 


r Schedule Drivers 

Units 

Development Start Year 1961 + 
Engineering Mgmt 1 6 to 100 


Low Most Likely 


Reusable ’'yes/no 

Crewed ’'yes/no 

Post-Apollo Man-Rated ''yes/no 
Parallelization ’'yes/no 


yes 


yes 


yes 


yes 


High 


2007 

2007 

2007 

25 

25 

25 


Analyze 


Funding Availability ' , 25 to 75 

75 

75 

75 

Manufacturing Methods ^ 6 to 100 

50 

50 

50 





Power Generated ’'Watts 

12000 

12000 

12000 










Design 

Manuf 

Duration [months} 

p 

24.5 

26.9 


c r 

31.54 

63.67 


p = 

3.17096 

3.24729439 


Q = 

0.22674 

0.3009655 

Outputs of Lognormal: 

Design 

Manuf 

Total 

Mean 

24.5 

26.9 

51.4 

Median 

23.3 

25.7 

50.4 

Mode 

22.6 

23.5 

43.6 

cr 

5.6 

3.3 

10.0 

5th 

16.4 

15.7 

36.7 

10th 

17.3 

17.5 

39.4 

20th 

19.7 

20.0 

42.9 

30th 

21.2 

22.0 

45.6 

40th 

22.5 

23.3 

43.0 

60th 

25.2 

27.3 

52.9 

70th 

26.3 

30.1 

55.3 

80th 

23.3 

33.1 

59.3 

90th 

31.9 

37.3 

64.6 

95th 

34.6 

42.2 

69.3 


Total ^ 
51.4 
100.21 

3.92028 

0.19308 


Outputs of Normal: 



Design 

Manuf 

Total 

Mean 

24.5 

26.9 

51.4 

Median 

24.5 

26.9 

51.4 

Mode 

24.5 

26.9 

51.4 

cr 

5.6 

3.3 

10.0 

5th 

15.2 

13.3 

34.9 

10th 

17.3 

16.3 

33.5 

20th 

19.7 

19.9 

42.9 

30th 

21.5 

22.6 

46.1 

40th 

23.0 

24.3 

43.3 

60th 

25.9 

29.0 

53.9 

70th 

27.4 

31.3 

56.6 

30th 

29.2 

33.9 

59.3 

90th 

31.6 

37.5 

64.2 

95th 

33.7 

40.5 

67.3 



Summary Results 


Tabular Results 


Graphical Results 







Conclusion 


• Objective of this task was to investigate 
feasibility of SERs 

- Valid SERs have been generated & applied in 
existing joint confidence level analyses 

- Statistically significant results achieved 

- SERs employed in a model for immediate use 

• Future work 

- Integrate into future version of NAFCOM 

- Refine SERs with new missions, additional effects 





SERRA Model 



• Schedule Estimating Relationships Risk 
Assessment (SERRA) model available for 
distribution 

• Excel-based implementation of SERs 

• Contact George Culver 
(george.a.culver@saic.com) for a copy 




SUPPORTING DATA 



J 4 Earth Orbiting SER 
Regression 


Start _ CDR _ Dur = 69.274[(Z>y Wt - 50) / 5000 + ( Numlnst - 1) / 1 2 + 
MaxData / 1024]® 203 TestAppr~° m ManufMethods~° m StartYr 0206 
DesignLife l3 ' n StreamEM~ 02m Observatory' 3 ' 12 ' 3 CommSat" 443 Military'* 1 *' 5 

F Test p-value = <0.001 
Pearson R 2 = 0.826 
Est Std Error = 0.329 

CDR _ Delivery Dur = 0.55 \EngrMgmt om Start Yr ° 555 
DesignLife 332 ' 2 Observatory 33 ' 53, CommSat~ om Military' 33 ' 01 Bus' 044 ' 

F Test p-value = <0.001 
Pearson R 2 = 0.856 
Est Std Error = 0.306 





\ Planetary SER 
* Regression 


Start _ CDR _ Dur = 0.' 7 59 FundAvail 0429 StartYr 0337 
DesignLife 0229 StreamEM^ 9393 RTG 0399 

F Test p-value = <0.001 
Pearson R 2 = 0.804 
Est Std Error = 0.227 

CDR _ Delivery Dur = 5 ,219[(DryWt -100)/ 4000 + (Numlnst - 1) / 1 2 + 
MaxData / 256)]° 065 StreamEM -9324 StartYr 96 ' 3 RTG 9316 

F Test p-value = <0.001 
Pearson R 2 = 0.751 
Est Std Error = 0.301 





EO SER Residual 
Analysis — Acceptable 



EO Design SER 
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Residual Analysis: Fitted Vs. Resid 
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Equal Variance Assumption: No significant trend 
evident, assumption valid. 


EO Manufacturing SER 


n q 

Residual Analysis: Fitted Vs. Resid 
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Equal Variance Assumption: Slight decreasing trend 
evident (cone), however assumption valid. 


EO Design Resid 




Normality 
Assumption: 
Log residuals 
normally 
distributed. 


Normal Quantile Rot 




Normality 
Assumption: 
Log residuals 
normally 
distributed. 



PL SER Residual 
Analysis — Acceptable 


PL Design SER 


Residual Analysis: Fitted Vs. Resid 
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Equal Variance Assumption: No significant trend 
evident, assumption valid. 



Normality 
Assumption: 
Log residuals 
normally 
distributed. 



Equal Variance Assumption: Slight decreasing trend 
evident (cone), however assumption valid. 



Normality 
Assumption: 
Log residuals 
normally 
distributed. 


Normal Quantie Rot 


